Email messaging has been an active field for nearly twenty years, leading to today's leading messaging tools and services including:                1. AOL Messenger,        2. Yahoo! Messenger,        3. MSN Messenger,        4. Skype.        
All of the existing messaging systems deal with one or both of the two standard live sensor feeds (microphone, webcam). In addition, they have very limited computer graphics associated with them: simple avatars, smiley faces. There are no stored sensor data used in today's applications. There is no composition of multiple, multidimensional, or mixed signals (live, stored, etc.). There is definitely no Digital Composition Codebook, as per our development below. From our perspectives, these early applications are just scratching the surface of what is to come, but can't predict or even suggest the future of rich media multisensory communications. Nevertheless, rich communications can be enabled with technology we have today, by dramatically altering the way in which messages are constructed and delivered, our main advances.
FIG. 1 illustrates the current generation messaging tools: Skype (for voice only), and Yahoo! Messenger for voice, video, and text (as displayed). The limit of computer graphics presented in these tools is the tiny presence of the smiley faces near the bottom of the Instant Message screen. There are no stored videos of sound clips, and there definitely are no tools for the composition of a multimedia message (much less a multisensory message).
FIG. 2 illustrates the current generation of smilies, avatars, and icons. These are generally a composite of several images (often just two), stored in the “gif” format, which permute (“animated gifs”) to give an impression of action. These are the first inklings of a future use of stored graphics in rich media messaging, as we envision it. Today, these are limited by bandwidth and computational resources at either end of the communication—bottlenecks we overcome.
FIG. 3 illustrates a more sophisticated structure, called 3D icons/avatars, being wireframe models that can be manipulated, are currently available as desktop add-ons (for example in screensavers), and sometimes for chat rooms, but are not generally part of messages—they require significant additional tools that are not commonly available. The creation of tools for the efficient representation and manipulation of multi-signal objects, and a method for their widespread availability for enhanced communications, are also part of this invention.
The existing messaging systems such as AOL, Yahoo!, MSN, and Skype, permit the creation and transmission of messages based on the use of microphones, webcams, and text, as well as file inclusions—all of which we will refer to as direct entry data (DED). The sensory data today is limited to sight and sound, and only that which is live-fed. In addition, there is only a hint of additional use of computer-generated effects (simple avatars, smiley faces, etc). In fact, the main purpose of these additional computer-generated effects has been to represent a person in case the video signal can't be transmitted, a situation that is common since it is quite difficult to transmit video in today's limited bandwidth environment.
In experiments we have conducted as of the date of this filing, we generated video at less than one frame per second—i.e., these were only still images! At worst, the images froze altogether. This bandwidth paralysis, complicated by the need for high computing resources for the management of compression/decompression and channel negotiation, which are still not being met by the latest generation Intel Pentium processors, may be hampering the vision of developers currently in this field. As we look ahead, we see an era when bandwidths, resources (especially at central server farms), and the opportunity to communicate richly will be ever present and in high demand. There is a need, however, for methods of performing such functions on common communications devices which will be functionally limited for the foreseeable future (e.g., cell phones).